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MAC with Action-Dependent State Information 

at One Encoder 

Lior Dikstein, Haim H. Permuter and Shlomo (Shitz) Shamai 

Abstract 

Problems dealing with the ability to take an action that affects the states of state-dependent communication 
channels are of timely interest and importance. Therefore, we extend the study of action-dependent channels, which 
until now focused on point-to-point models, to multiple-access channels (MAC). In this paper, we consider a two-user, 
state-dependent MAC, in which one of the encoders, called the informed encoder, is allowed to take an action that 
affects the formation of the channel states. Two independent messages are to be sent through the channel: a common 
message known to both encoders and a private message known only to the informed encoder In addition, the informed 
encoder has access to the sequence of channel states in a non-causal manner Our framework generalizes previously 
evaluated settings of state dependent point-to-point channels with actions and MACs with common messages. We 
derive a single letter characterization of the capacity region for this setting. Using this general result, we obtain 
and compute the capacity region for the Gaussian action-dependent MAC. The unique methods used in solving the 
Gaussian case are then applied to obtain the capacity of the Gaussian action-dependent point-to-point channel; a 
problem was left open until this work. Finally, we establish some dualities between action-dependent channel coding 
and source coding problems. Specifically, we obtain a duality between the considered MAC setting and the rate 
distortion model known as "Successive Refinement with Actions". This is done by developing a set of simple duality 
principles that enable us to successfully evaluate the outcome of one problem given the other 

Index Terms 

Actions, binning, channel capacity, channel coding, dirty paper coding, duality between channel coding and source 
coding, Gel'fand-Pinsker channel, non-causal side information, rate distortion, successive refinement with actions. 

I. Introduction 

STATE-DEPENDENT channels model a communication situation where the channel is time variant. Such channels 
characterize a significant collection of communication scenarios, ranging from interfering transmissions models to 
cases where the states are generated by nature. Problems of coding for these channels have received much attention 
due to the wide range of their potential applications. These applications vary from modeling communication links 
such as fading, to interference in a wireless network. Furthermore, problems relating Multiple Access Channels 

'Parts of this paper appeared in the IEEE International Symposium on Information Theory (ISIT 2012), Cambridge, MA, US, July 2012 and 
at the IEEE 27th Convention of Electrical and Electronics Engineers in Israel (lEEEI 2012), Nov. 2012. 
^This work has been supported by the CORNET Consortium Israel Ministry for Industry and Commerce. 
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(MAC) with channel state information (CSI) have been thoroughly studied due to their importance in modeling 
wireless communication systems. Most of the channels investigated until now have been examined under the 
assumption that the states affecting the channel cannot be influenced by the communication system. 

In this paper, we consider an action-dependent channel. The motivation for studying these channels stems from 
the implications and practical use of the 'action' in modeling important communication scenarios. For instance, one 
interpretation of the action could be a noisy public relay. In this case, the relay outputs are modeled as a function of 
the messages. We provide a relay output sequence to be transmitted: A" {Mi, M2) and obtain the state sequence, S"", 
via the memoryless noisy transformation p{s\a). The relay outputs are public and, therefore, monitored beforehand 
so that S"" is known at the transmitter Our model is the natural extension of that for the two-user cognitive setting. 

So far the most studies on action-dependent channels focused on point-to-point channels. In this work, we broaden 
the research of action-dependent channels to MACs. We consider a MAC where one of the encoders is allowed 
to take an action that affects the formation of the channel states. Specifically, we examine a MAC communication 
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Fig. 1. The MAC with action-dependent state information at one encoder, which is considered in this paper 



system, that is illustrated in Fig. [T] where two encoders have access to a common message and only one, the 
informed encoder, has access to a private message. This encoder can generate an action sequence dependent on 
both messages that, in turn, affects the channel states. Furthermore, the states affected by the action sequence are 
accessible non-causally to the informed encoder when producing the channel input. We will refer to this model 
as the Action-MAC. We characterize the capacity region of this channel for the general finite alphabet case with 
a single letter expression. This is done by using a random coding scheme and a binning technique, where the 
encoding of the messages is done in three parts. In the first part, the uninformed encoder encodes a message using 
its signal, Xi. In the second part, the informed encoder encodes a message using the actions where Xi is used as 
side information. In the third stage, a Gel'fand-Pinsker coding is done from the informed encoder where the action 
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and Xi are used as side information. 

The study of state-dependent channels dates back to Shannon [I], who first introduced and characterized the 
capacity of a state-dependent, memoryless point-to-point channel with independent and identically distributed (i.i.d.) 
states available causally at the encoder Gel'fand and Pinsker 0, and later Heegard and El Gamal (O], studied a 
case in which the encoder observes the channel states non-causally. They derived a single letter formula for the 
capacity, using a binning coding scheme. The main idea of this scheme is to generate a subcodebook for each 
message. Next, in order to send a message we send one of the codewords in the subcodebook that is jointly typical 
with the sequence of channel states. The results obtained by Gel'fand and Pinsker were used by Costa in his famous 
"Writing on Dirty Paper" He applied these results to the case where there are two additive Gaussian noise 
sources, where one of the noises represents the interferences and is modeled by the channel's states. 

An extension of this model, known as the Generalized Gel'fand-Pinsker (GGP) MAC, was studied by Somekh- 
Baruch, Shamai and Verdu, in their paper ||5]. They considered a MAC with common and private messages where 
the encoder informed of the private message is additionally informed of the channel states non-casually. They 
characterized the capacity region for the general finite alphabet case using a generaUzed binning coding scheme. A 
MAC with private messages at both encoders and state information known to one encoder channel was examined in 
161, where an inner bound for the capacity region in the general discrete memoryless case was found. Further MAC 
models, where the states are known causally or strictly causally were considered by Lapidoth and Steinberg in ||2], 
ISl and later by Li, Simeone and Yener in ||9l. Another version of a MAC, where the states are known noncausally 
at one encoder and causally at the second encoder, was considered by Zaidi, Piantanida and Shamai in ifTol . 

The novel idea of an action-dependent state scenario was introduced in the work of Wiessman ifTTI . In his paper, 
he considered a point-to-point channel where the encoder is allowed to take an action that can affect the channel's 
states. He characterized the capacity for the case where the channel inputs are allowed to depend causally or non- 
causally on the state sequence. Furthermore, the Gaussian case for this channel was also introduced, however, only 
an achievable scheme was given and the capacity for this case remained unsolved. 

In this work, we consider a MAC with a common and private message along with CSI at one encoder However 
in our setting, this encoder can now take an action that affects the states. This generalizes the results presented in 
the works of Somekh-Baruch, Shamai and Verdu ||5] as well as Wiessman ifTTI . We particularly focus on analyzing 
the Gaussian case for our channel model. We find the capacity region and compute it. In the process, the new 
results obtained help us find the capacity expression for the Gaussian point-to-point action-dependent channel left 
open in ifTTl . A similar result was also derived by obtaining a correspondence between the action-dependent point- 
to-point channel to the GGP MAC with only a common message. This same correspondence was also obtained 
simultaneously and independently by Choudhuri and Mitra in lfT2ll . Furthermore, we investigate the dual relationships 
between channel coding and source coding problems with actions. We establish a duality between our Action-MAC 
to the rate distortion model known as "Successive Refinement with Actions", presented in |[T3l and explore the 
similarities between them. We show how using the results gained from the solution of the Action-MAC assists 
us when analyzing the "Successive Refinement with Actions" setting and vice versa. We obtain a set of duality 
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principles which help predict the outcome of one problem given the other. These set of principle also help establish 
more dualities between known channel coding and source coding problems with actions. 

The remainder of the paper is organized as follows. Section |II] presents the mathematical notation used in this 
paper and the exact formulation of the Action-MAC setting. In Section |III1 we state our capacity results, which 
include the capacity region of the Action-MAC, and discuss additional special cases that show consistency with 
previous works. Sections |IV] and |V] are devoted to describing the achievability coding scheme and the converse 
proof respectively. Section |VT] examines the Gaussian case for the Action MAC, where we find the capacity region 
for this specific setting and compute the region for different parameters. Here, we also find the capacity expression 
for the point-to-point model. In Section IVIII rate distortion coding duals are considered. We conclude in Section 
IVIIII with a summary of this work. 



Throughout the paper, random variables will be denoted by upper case letters, deterministic realizations or specific 
values will be denoted by lower case letters and calligraphic letters will denote the alphabets of the random variables. 
Let a;" denote vectors of n elements, i.e. a:" = {xi ) and xl denote the i - j + 1-tuple {xi,Xi+i, Xj) 

when j > i and an empty set otherwise. The probability distribution function of X, the joint distribution function 
of X and Y and the conditional distribution of X given Y will be denoted by Px, Px,y' and Px\y^ respectively. 

We consider a channel coding MAC with action-dependent states, where the states are known non-causally at one 
encoder, as illustrated in Fig. [T] The Action-MAC setting consists of two transmitters (encoders) and one receiver 
(decoder). Let n denote the block length and A,S,Xi,X2 and y be finite sets which denote the actions, states, 
the uninformed encoder's inputs, the informed encoder's inputs, and the outputs, respectively. A state information 
channel is described by a triple {A, Ps\AjS) and is assumed to be memoryless with transition probabilities: 



Definition 1: A ((2"^^ , 2"^^), n) code for the channel in Fig. [T] consists of two sets of integers, Aii = 
{1, 2, 2"^i} and Ai-z = {Ij 2, 2"^^}, called message sets. An index is chosen uniformly and independently 
by the senders out of the message sets. The uninformed encoder selects a channel input sequence, X" — 
Given the messages Mi, M2, an action sequence denoted A" = A"(Mi, M2) is selected by the informed encoder 
A state sequence, 5", is then selected by the channel, with being the input chosen by the informed encoder 
Next, a channel input sequence, X2 ~ (A/i, M2, 5"), is selected. The output of the channel is denoted F". 
The channel is characterized by the conditional probability p{yi\xi^i,X2,i,Si) and is assumed to be memoryless. 
Therefore, both probabilities do not depend on the index i, i.e. 



II. Notation and Problem Definition 



p{si\a\s'' \mi,m2) =_p(sj|ai). 



(1) 



p{yi,Si+i\x\,X2, s') = p{yi\xi,i, X2,j, Si)p(si+i). 



(2) 



The code is defined by the encoding functions: 



•n 



(3) 
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and 

f2:MixM2xS'' ^ X^, (4) 

an action encoder 

fA:MixM2^ ^" (5) 

and a decoding function 

g -.y" ^ {Mix M2). (6) 
We define the average probability of error for the ((2"-^i,2"-^2),n) code as follows: 

^i"^ - 2»(^i+^^) ^ Pr{g(y")^(™i,™2)|(mi,m2)sent}. (7) 

"11 ,"12 5 All xM2 

A pair rate (i?i, i?2) is achievable if there exists a sequence of codes (2"-^i , 2"^^ , n) s.t. Pi"^ ^ 0. 
The capacity region is the closure of all achievable rates. 

III. Capacity Results for the Action-MAC 

A. Capacity Region 

The following theorem provides an expression for the capacity region of the MAC with action-dependent state 
information at one encoder channel, for finite alphabets A,S,Xi,X2: 

Theorem 1: The capacity region of the MAC with action-dependent state information at one encoder, as shown 
in Fig. [T] is the closure of the set that contains all the rates that satisfy 

R2 < I{U;Y\Xi)-I{U;S\A,Xi) 
R1+R2 < I{Xi,U;Y)-I{Xi,U;S\A), (8) 

for some joint probability distribution of the form 

Pa,SM,Xi,X2,Y = PxiPa\XiPs\aPu\s,a,XiPx2\XuS\uPy\XuX2.s (9) 

and \U\ < \A\\S\\Xi\\X2\ + 1. 

Lemma 2: The capacity region described in Theorem 1, given in dHJ, is convex. 
The proof of Lemma |2] is given in appendix |A] 

In the following corollaries we present an alternative representation for the capacity region given in Theorem [T] 
which provides a more intuitive view. Furthermore, we derive the corner points for the Action-MAC rate region 
and the distribution of rate resources. 

Corollary 3: The following region is equivalent to the one presented in Theorem 1: 

R2 < I{A,U;Y\Xi)-I{U;S\Xi,A) 
R1+R2 < IiXi,A,U;Y)-I{Xi,U;S\A). (10) 
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Proof: It is clear that the region ([TO) contains the one in © since I{U;Y\Xi) < I{A,U;Y\Xi) and 
I{Xi, U; Y) < I{Xi,A, U\ Y). For the converse, we have: 

I{A, U- Y\X^) - I{U- S\Xi,A) = I{A, U; Y\Xi) - I{A, U; S\Xi,A) 

= IiU;Y\Xi)~IiU;S\Xi,A), (11) 

taking U = {A, U), where (Xi, S, A, U, X2, Y) satisfies the same joint distribution relations as {Xi, S, A, U, X2, Y) 

■ 

Furthermore, notice that if we rearrange the expressions in ( fTOl l by using the chain rule and the Markov Xi — A — S, 
we can express the capacity region as: 

R2 < I{A;Y)-I{Xi;Y) + I{Xi,U;Y\A)-I{Xi,U;S\A) 

R1+R2 < I{A;Y)+I{Xi,U;Y\A)- I{Xi,U;S\A). (12) 

This result provides intuition, and makes immediate sense. First, the informed encoder transmits information 
using the action sequence A at rate I{A; Y). This first transmission is decoded and used at the decoder to decode 
a second transmission; hence the conditioning. Next, by Gel'fand-Pinsker given A: Xi and U can be decoded. 
Namely, we notice the same expression, I{A; Y) + I{Xi, U; Y\A) — I{Xi, U ; S\A), appears in the bound on R2 
as well as the bound on i?i + i?2- However, in the expression bounding R2 we omit the rate I{Xi; Y), which is 
the rate used by the uninformed user. 

Corollary 4: Another presentation of the capacity region in Theorem [T] is: 

R2 < IiA;Y\X,)+I{Y;U\A,X,)-I{S;U\A,X,) 

R1+R2 < IiXv,Y)+I{A;Y\Xi)+I{Y;U\A,Xi)^I{S;U\A,Xi), (13) 

which follows from a different reorganization of the region presented in Corollary |3]by using the chain rule together 
with the Markov property Xi — A — S. 

This is another intuitive perception for our rate region. Here, we can recognize that the uninformed user sends 
information at rate I{Xi; Y). This transmission is then decoded at the decoder and used as side information. Next, 
the informed user sends information about the action at a rate I{A; Y\Xi) (since Xi was decoded and is now known 
at the decoder - hence the conditioning). Finally, now that {Xi, A) are known at the decoder and hence conditioned 
upon, the rest of the information is sent in a Gel'fand Pinsker-like scheme at rate /([/; A) — I{U; 5*1X1, A). 

Namely, we can look at the conditioned expressions of the region as standard expressions of a "MAC with common 
message and state information known to one encoder" channel, presented in IS), i.e. 

R2 < I{U;Y\Xi)-I{U;S\Xi) 

R1+R2 < I{Xi,U;Y)- I{Xi,U;S) 

with a supplement of side information A. 

This presentation also helps us to recognize the corner points of our region. The region is illustrated in Fig. |2] 
The corner points R2) are presented in Table I] 
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TABLE I 

Corner points (_Ri , i?2) of the Action-MAC setting 



Ri 


R2 


I(Xi;Y) + /(A; Y\Xi) + I{Y; U\A, Xi) - I{S; U\A, Xi) 
I(Xi:Y) 




i{A; y|Xi) + i{U] y|Xi, A) - i{U\ s\Xi,a) 




Fig. 2. The rate region for the Action-MAC 



B. Special cases 

Before proving the theorem, let us examine some special cases in order to gain some insight and persuade 
ourselves that the following region is consistent with previous results. 

Case 1: The point-to-point channel with action-dependent states. For this case, we take Ri — 0, Pa\Xi — Pa, 
Pu\s.A-Xi = Pu\s.A, Px2\XuS,u = Px2\s,u- Hence, the region in Theorem [2 becomes: 

i?2 < I{U;Y)-I{U;S\A) 

for a probability distribution of the form PaPs\aPu\s,aPx2\s,uPy\X2,s- This result is the action-dependent point- 
to-point channel capacity discussed in lITTl . 

Case 2: MAC with a common message and a private message known to one encoder, i.e. \S\ = 1. For this case, 
we take U = X2 and we have I{U; S\A) = and I{Xi, U ; S\A) = 0. Furthermore, there is no point in allocating 
resources to the action sequence. As a result, we obtain the following region: 

i?2 < IiX2;Y\Xi) 
R1+R2 < I{Xi,X2;Y), (14) 
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for some joint probability distribution of the form PxiPx2\XiPy\Xi,X2- 

Case 3: Cooperative multiple-access encoding with states available at one transmitter 15]. Assume that we have 
a malfunction at the action encoder, i.e. we cannot choose an action that affects the formation of the states, but the 
Encoder 2 still knows the states noncausally. For this case ^ = 0; therefore, the following expressions, I{U ; S\A) 
and I{Xi, U ; S\A), become I{U ; S) and I{Xi, U; S), respectively. Hence, we have the following capacity region 

i?2 < IiU;Y\Xi)~IiU;S\Xi) 
R1+R2 < I{X,,U;Y)-I{X,,U;S), (15) 

where the probability distribution PxiPa\XiPs\aPu\s,a,XiPx2\Xi,s,uPy\XuX2,S reduces to 

PxiPsPu\s,XiPx2\Xi,s,uPy\Xi,X2,s- This is, indeed, the result of the Generalized Gel'fand Pinsker channel, 
introduced in ||5]. 

IV. Proof of Achievability 

In this section we prove the achievability part of Theorem [T] Throughout the achievability proof we use the 
definition of a strong typical set lfT4l . The set Te ^^\x, Y, Z) of e-typical n-sequences is defined by {(x", i/", z") : 
^\N{x, y,z\x"', y", z") — p(x, y,z)\ < e ■ p{x, y, z) V(x, y, z) e <Y x x Z}, where N{x, y, z|x", y", z") is the 
number of appearances of (x, y, z) in the n-sequence (x", y", z"). 

The main idea of the proof is based on a random coding scheme, where the encoding of the messages is done 
in three parts. First, the uninformed encoder transmits Xf(mi) from it's code-book at rate I{Xi;Y). Secondly, 
the informed encoder chooses an action sequence A". As a result, a state S*" is generated. The action sequence 
is then sent at rate I{A; Y\Xi), where Xi is treated as side information known at both the informed encoder and 
the decoder Next, the informed encoder can transmit at rate I{U;Y\A, Xi) — I{U; S\A, Xi) using a Gel'fand 
Pinsker scheme; i.e. choosing a codeword [/" from a subcode-book C(mi,m2), such that it is jointly typical with 
{X" , A" , S"), where (X", A") were already decoded and are now used as side information at the encoder and 
decoder. 

Proof: Fix a joint distribution of Pa,s,u,Xi,X2.y = PxiPa\XiPs\aPu\s,a.XiPx2\Xi,s,uPy\x,,X2,s where 
Py\Xi,X2.s is given by the channel. 

Code Construction: For the uninformed encoder, generate 2"^^ independent codewords, (mi), mi G 
{1, 2, 2"^^}, where each element is i.i.d. ^ Y[i=i Pi^i-i)- For the informed encoder, generate 2"(^i+^^) action 
sequences, A"(mi,m2) ^ IliLi -P('^«|2^i,0- addition, for each set of messages, (mi,m2), where mi x m2 G 
{l,2,...,2"(^i+^2)j.^ generate 2"(^i+^2) bins. Next, generate randomly 2"^ codewords, u(l), m(2), u(2"^), 
each according to ^ JliLi Pi'^ilO'i^ Distribute the codewords uniformly among the bins. Now, each message set 
(mi, m2) has a subcode-book denoted c(mi, m2) of 2"(^~(^i+^2)) codewords. The code construction is illustrated 
in Fig. 121 

Encoding: The uninformed encoder transmits X" (mi). Next, the informed encoder chooses an action sequence 
A" (mi, m2). As a result, a state, 5", is generated. Now, the informed encoder chooses a codeword [/"(mi, m2, k) 
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2"^i codewords Xf 




Fig. 3. Encoder 1 chooses the codeword X"{mi) from its code-book of size 2"^i . For each codeword, X"(mi), we have 2"«2 bins. We 
choose the second bin according to 7712 (notice that we have a total of 2"^i x 2"^^ = 2"(^i+^2) bins, one for each message set, (mi,m2)). 
Now, we look in our bin, (mi, 012), of 2"(^^(^i+^2)) codewords for a specific codeword, U"{m-i, 1712, k), such that it is jointly typical 
with {X^,S",A"). 



from bin (mi, 777,2) with the smallest lexicographical order such that it is jointly typical with (Xf , S", A"), i.e. 

(C/"(mi,m2,fc),Xi"(mi),5",A") er,(")(C^,^i,^,^)- (16) 

If such a codeword, C/", does not exist, namely, among the codewords in the bin none is jointly typical with 
(X", S"", A") , choose an arbitrary [/" from the bin, U"{k) G C(mi, 777,2) (in such a case the decoder will declare 
an error). The input sequence to the channel, X'2, is according to ~ p{x2\xi, s,u) and the encoder transmitter 
transmits X2.i at time i e [1,11]. 

Decoding: The Decoder looks for the smallest value of ttii, 7712 for which there exits a k such that: 

(A"(77ii,m2),t/"(mi,m2,^),Xr(^i),^") e'T;^") (17) 

If none or more than one such triplet is found, an error is declared. The estimated messages sent are 7711 and 7772. 
Analysis of the probability of error: 

Without loss of generality, we can assume that messages (7771,7712) = (1, 1) were sent. 
An error occurs in the following cases. We define the events: 

El = {yU" eC{l,l),{X^,W\S'\A'') iT}''HXi,U,S,A)}, (18) 
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E2 = {vc/" e c(i,i),(A",Xi",!7",y") ^r,(")(AXi,[/,r)}, (i9) 

i?3 = {3m2^1:(A"(l,m2),Xr(l),[/"(l,m2,fc),r")eri"'(A,Xi,[/,r)}, (20) 

Ei = {3mi^l:(A"(mi,l),X{'(mi),t/"(mi,l,fc),y")er/"nA^i,C/,>^)}, (21) 

£.5 = {3nn^l,?n2^1: («"(^i>2),^r(mi),C/"(mi,m2,fc),y")er,(")(A^i^ (22) 
Then, by the union of events bound: 

Pi") Pr(Pi u £:2 u ^3 U ^4 u £:5) 

< P{Ei) + P{E2) + P{E3) + PiEi) + P{Ez). 

Now, let us find the probability of each event: 

• For the first error, note that we have 2"(^^(^i+^^)) codewords in each bin. In addition, the probability 
of (Xf (mi), yl" (7711,7712)) e T^''\Xi,A) is almost surely 1, and 5" is distributed - p(s"|a", xi') = 
n"=i -P(*iki)- Therefore, if the number of codewords in each bin is bigger than 2"'^'-^''^^-^^'^\ namely 
R — (i?i + R2) > I{U ; 5*1X1, A), then, according to the covering lemma lfT4l . at least one codeword, [/", is 
jointly typical with (X", S", A") with high probability. Hence, if the number of codewords in each subcode- 
book is more then 2"'f('^;^^l^i.^), then the error P{Ei) — )- as n — 00. Thus, 

Pr((J7"(l,l,fc),Xi",5",^"(l,l)) e r,(")|Xi"(l),^"(l,l) € 7;(")(C/,Xi,S',A)) = 2-"-f(^'^l-^i'^), 

therefore for R - {Ri + R2) > I{U; S\Xi,A) and l<k< 2"(J?-(«i+fl2))^ the probability 

Pr ( (c/"(i, 1, fc), Xi"(i), 5", A"(i, 1)) i r/")|xr(i), ^"(1, 1) e r,(")) ^ 0. 

• Consider the second error: 

p(S2) = Pr((A",Xi,t/",r") ^ r/")). 

As for the first event, if we have R - [Ri + R2) > I{U; S\Xi, A), then the probability of 
(^",J7",X{\S'") € tJ'''\u,Xi,S,A) approaches 1. Furthermore, due to the Markov Lemma [15] 
and the Law of Large Numbers, as well as the fact that y" is distributed according to ^ Py\Xi,X2,s^ the 
probability of (^", t/", S"", X]", X^^, y") e tJ-^^U, Xi, S, A,Y) is almost surely 1. Hence, P(£;2) ^ as 

71 — >■ 00. 

• For the third error, consider: 

P(i?3) = Pr ((^"(1, 7772), Xr(l), t/"(l, 7772, k),Y-) € Tj^HA, X,,U, Y)) . 

To bound the probabiUty of this event, note that: 

Pr((A"(l,r772),Xi"(l),[/"(l,77i2,A:),y") S T}'''' {A, Xi,U,Y)\m2 ^ l) < 2-"^(^'^'^l^i) 
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2~^-i(u-x\x,) ^ (23) 



Therefore, using the union bound: 



Pr 



(u4(A"(l, m2), 1112, k),Y^) e r/") {A, Xi,U, Y)}\m2 ^ l) < 2-"(^(^'^l^i)-(«-(«i+^=))). 

(24) 

Let us take R — {Ri + R2) = I{U; S\Xi, A) + e. It follows that the probability where 7712 7^ "^2 for 
which there exists a fc such that (yl"(l, j), (1), fc), F") € re^"^(A, Xi, JJ, F) vanishes if we have 

R2 < I{U- Y\Xi) - I{U; S\A, Xi) - e. 
For the forth error, consider: 

P{Ei) = Pr ((A" (mi, 1), Xr(mi), f/"(mi, 1, fc), F") S T,'") Xi, iJ, F)) . 

To bound the probability of this event, note that: 

Pr((A"(mi,l),Xr(mi),C/"(mi,l,fc),y") er,(")(A^i,C/,y)l^i 7^ l) < 2-"^(-4'^i'^'^) 

< 2-"^(^i'^'^). (25) 

Therefore, using the union bound: 

Pr (Ufc {(A"(mi,l),Xr(mi),C/"(mi,l,fc),y") e T}''\A, Xi,U,Y)}\mi ^ l) < 2-"(^(^'^i.t^;^)-(«-(«i+^-))) 

_ 2-"(-'"(^i.'^;'*')--'"('^;-5|Xi,A)-6) 

(i^) 2-n(-f(^i,C/;'i')--f(^i,C/;S|A)-€) 

so it follows that the probability that there exists an mi 7^ mi for which there exists 
a k such that 1), 1, fc), F") G Te'-''\A, Xi,U,Y) vanishes if we have 

Ri<IiXi,U;Y)-I{Xi,U;S\A). 

For the fifth term, consider: 

P{E5) = Pr ((yl"(mi, m2), X^{mi), [/"(mi, m2, k), F") S T^^"' (^, Xi, [/, F)) . 
To bound the probability of this event note that: 

Pr ((A"(mi,m2),Xi"(mi),C/"(mi,m2,fc),r") G %^"\A, Xi,U,Y)\mi ^ l,m2 ^ l) < 2-"^(^'^i'^^^) 
Therefore, using the union bound: 

Pr (ufc {(A"(mi,m2),Xr(mi),C/"(mi,m2,A:),r") G %^''\A, Xi,U,Y)}\mi ^ l,m2 ^ l) 

^ 2-"(-f(-^i^'^;^)-(^-(«i+-R2))) 
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.-n(I(Xi,U;Y)-I{Xi,U;S\A)-e) 



SO it follows that the probability that there exists an ifii ^ mi and 1112 7^ m2 for which there exists a k such 
that {A''{l,j),Xl'{l),W{l,j,k),Y''') G T}"Ha, Xi,U,Y) vanishes if we have Ri + R2 < I{Xi,U;Y) - 
I{XuU;S\A). 

{*) Follows from Corollary 1. 

Follows from the Markov chain Xi — A — S. 
Combining the results, we have shown that P{E) ^ as n ^ 00 if 



The above bound shows that the average probability of error, which, by symmetry, is equal to the probability for 
an individual pair of codewords, (7TI1, 7712), averaged over all choices of code-books in the random code construction, 
is arbitrarily small. Hence, there exists at least one code, ((2"^^, 2^^'^), n), with an arbitrarily small probability of 



In the previous section, we proved the achievability part of Theorem [T] In this section, we provide the upper 
bound on the capacity region of the MAC with action-dependent state information at one encoder, i.e. we give the 
proof of the converse for Theorem [T] 

Proof: Given an achievable rate, {Ri,R2), we need to show that there exists a joint distribution of the form 

PxiPa\XiPs\aPu\s,a,XiPxi\Xi,s,uPy\XuX2.s such that 



Since (i?i,i?2) is an achievable pair-rate, there exists a code, (tt,, 2"^% 2"^^), with a probability of error, Pe , 
arbitrarily small. By Fano's inequality. 



R2 < IiU;Y\Xi)- IiU;S\Xi,A) 



R1+R2 < I{Xi,U-Y)~I{Xi,U;S\A). 



error 



V. Proof of Converse 



i?2 < I{U;Y\Xi)-I{U;S\A,Xi) 



R1+R2 < I{Xi,U;Y)-I{Xi,U;S\A). 



7J(Mi, Mair") < n{Ri + R2)Pt^ + H{P^^^) ^ 



(26) 



so we can say that e„ — > as Pe — >■ 0. Furthermore, 



F(M2|Mi,y") < i7(Mi,M2|y") < e, 



(27) 



To bound the rate R2 consider: 



nR2 = H{M2) 



(a) 



H{M2\Mi) 
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= H{M2\Mi) - i/(M2|Afi,r") + i7(Af2|Mi,i^") 
= /(M2; r"|Mi) + i7(M2|Mi, r") 

(6) 

< /(Af2;y"|Afi)+ne„ 
'='/(M2;y"|Mi,Xr(Afi))+ne„ 

/(A/2; y"|A'/i, - /(A/i, A/2; A") + ne„ 



HM2; y.|Mi, r'-i) - /(A/i, M2; SIVi, A") + ne„ 

n 

^ /(A/2, siVi- ^'.IMi, xr, - /(siVi, A"; r,|xr, M2, 

- /(A/i, A/2, r^-1; ^r+i, + ^(>"'"'; ^"|Mi, M2, 5IVi, A") + ne„ 

n 

J2 IiM2, SIVi, A"; r,|A/i, Xi", - /(A/i, A/2, y^-i; Sf+i, A") + ne„ 

i=l 
{h) " 

< ^ /f(F,|^i,.) - /f (r.l^r, A/i, M2, 5;Vi, A") 

1=1 

^ [/f A") - H{S,\X^, M,,M2,Y^-\S:+„A")] + ne„ 

r?, 

(i) 

i=l 
n 

= ^ liY,; C/,|Xi,,) - /([/,; A,) + ne„. 



^/r(y,|Xi,,) ~ /f(y,|Xi,„c/,) - [h{s,\Xi,,,a,) - h{s^\Xi,^,u^,a,)] 



where 

(a) follows from the fact that Mi and A/2 are independent, 
(&) follows from Fano's inequality, 

(c) follows from the fact that (A/i) is a function of A/i, 

(d) follows from the fact that (A/i, A/2, - A" - S"" form a Markov chain, 

(e) follows from the chain rule, 
(/) follows from the fact that 

/(A/2; r,|A/i, , r'-i) = /(A/2, sr+i, ^"; >"»|A/i, Xi", y^-i) - /(sr+i, i^^a/i, A/2, x^", r^-i) 

(chain rule) and /(A/i, M2; 5"|A/i, ^f+i, A") = /(Mi, M2, F^-i; X^", A") - 

I{Y'-^;Si, ^"|A/2, Sp+i, A/i, Xi", A") (chain rule and Markov relation), 

(g) follows from the fact that 

Er=i ^(^^"+1. ^^\M2, A/i, xr, y-i) = Er=i ^(^'"'; ^'.^ ^"|Af2, A/i, xr, 5r+i, 

due to the Csiszar sum identity, 

(h) follows from the definition of mutual information and from the fact that conditioning reduces entropy, 

(i) follows from the choice of Ui = (Xj"\ A/i, A/2, S^+i, F^-^, and the Markov chain 

Si — Ai — ('5'^Yi 1 ^i-i I ^r+i 1 X^). 
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Hence, we have: 

1 " 

i?2 < - y[/(r,;C/,|Xi,,)-/(C/.;5,|Xi,„A0] + e„. (28) 
n ^ — ' 

i=\ 

To bound the sum of rates, R\ + R2, consider: 

+i?2) = H{Mi,M2) 

= H{Mi,M2) - H{Ah,M2\Y") + i?(A/i, Afal^") 
= /(Ml , M2 ; r " ) + ( Af 1 , A//2 1 r " ) 

< /(Afi,A//2;y")+"en 

/(A'/i, Xr(Afi), A/2; F") + ne„ 

/(Xr, Afi, A./2; r") - /(Xr, Afi, Af2; + ne„ 

n 

Y,HX^,Mi,M2;Y,\Y'-')~I{X^,Mi,M2;S,\S:+„A")+nen 

n 
i=l 

- I{X^, AIi,M2,Y'-^;S,\Sl\„A^) + I{Y'-^;S,, A^\X^, A/i, A/2, Sr+i, + ne„ 

n 

1=1 

< //(y.) - H{Y,\x^, A/i, A/2, 5iVi, - [/r(5,|sr+i, A") 

i=l 

- H{S,\X^, A/i, A/2, ^IVi' ^")] + nen 

n 

t^Y,H{Y,) - H{Y,\Xi^„U,) - [H{S^\A,)-H{S,\U,,Xi^,,A,)]+ner, 

i=l 
n 

= I{Y^;Xi,„ Ui) - /(Xi,„ U,- S,\A,) + ne„, 
i=i 

where 

(a) follows from Fano's inequality, 

(6) follows from the fact that X"(A/i) is a function of Mi, 

(c) follows from the fact that (X", A/i, A/2) - A" - S"" form a Markov chain, 

(d) follows from the chain rule, 

(e) follows from the fact that /(Xj", A/i, A/2; Ki|F'-i) = /(Xf, A/i, A/2, S'f+i, A"; FilF^-i) - 
/(5JVi,^";K,|Xi",Mi,A/2,r'-i) (chain rule) and /(X^ Mi, A/2; ^^l^f+i, 

/(Xi", Ml, M2, r*-i; S-ilSp,^!, ^") - /(F'-i; S-i, yl"|Xi", A/i, A/2, Sf+i, A") (chain rule and Markov chain), 
(/) follows from the fact that 

Er=i ^(^,"+1, Ml, A/2, r^-i) - Er=i ^"l^r, ^i, A/2, Sf+i, ^") 

due to the Csiszar sum identity. 
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(g) follows from the definition of mutual information and from the fact that conditioning reduces entropy, 

(h) follows from the choice of U, = {Xl~\ X^^^^^, Mi, M2, Sl\^,Y'-^ , A"-) and the Markov chain 

Si — Ai — {Sl'-_^_i, Ai-i, A^_^^). 
Hence, we have: 

1 " 

R1 + R2 <-y[IiY,;Xi,„U,)~I{Xi,„U,;S,\A,)]+en. (29) 

i=l 

The expressions in (|29] l and (|28l l are the average of the mutual informations calculated from the empirical 
distribution in column i of the code-book. We can rewrite these equations with the new variable, Q, where Q = 
i € {1, 2, n} with probability i. The equations become: 



n 

n ^ — ' 

1=1 

1 " 

= - J2^IiYQ; Uq\Xi^q, Q = i)- I{Uq- Sq\Xi^q, Aq,Q = i)] + e„ 



n 

i=l 



= I{Yq; Uq\Xi^q, Q) - I{Uq; 5q|Xi,q, Aq,Q) + e„ 

= HYq; Uq\Xi,q, Q) - /(C/q, Q; Sq\Xi,q,Aq) + I{Q- Sq\Xi,q,Aq) + e„ 
<I{YQ-UQ,Q\Xi^Q)~I{UQ,Q-SQ\Xi^Q,AQ) + er,, 

and similarly, 

1 " 

Ri+R2<- V[/(r,; Xi,,, [/,) - U,-S,\Ai)\ + e„ 

n ^-^ 

i—i 

1 " 

= - Y}^{Yq-Xi^q, Uq\Q = z) - /(Xi^Q, Uq- Sq\Aq,Q = i)] + e„ 



n 

i=l 



= /(Fq; L/qIQ) - /(Xi,Q, [/q; ^q|Aq, Q) + e„ 

= /(Fq; Xi,Q, t/glQ) - /(Xi,Q, I/q, Q; 5q|^q) + /(Q; 5q|^q) + e„ 

< /(Fq; Xi,Q, C/q, Q) - /(Xi^Q, Uq, Q- Sq\Aq) + e„, 

where the last step foUows from the fact that I{Yq; Xi q, Uq\Q) < I{Yq; Xi q, Uq, Q) and the stationarity of 5*^. 
Let us denote Xi = Xiq, U = {Uq,Q), Y = Yq, S = Sq, A = Aq. Now, taking the limit as n — > 00, 

(n) 

Pe 0, we conclude: 

i?2 < IiU;Y\Xi) - I{U;S\Xi,A) 

R1+R2 < I{Xi,U;Y)~I{Xi,U;S\A). 

Next, we want to prove the cardinality bound on U. Using the support lemma introduced in lfT6l . U needs to 
contain |^| |iS| jA"! | IA2I — 1 elements in order to preserve the joint distribution Pa.s,Xi,X2- In addition, we need 
two more elements to preserve the expressions H{Y\U, Xi) and H(S\Xi, A, U). Further, from the markov relation 
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U — {Xi,X2,S) — Y, we can determine that the joint distribution Py,s,Xi,X2 is also preserved. Therefore the 
cardinality of U is bounded by |^||5||<fi||Af2| + 1. 

Finally, we will prove the following Markov relations to complete the converse; 

1) Ps\A,xAs\a,xi) = Ps\A{s\a). 

p{s\a,xi)= ^ p{s,mi,m2\a,xi) 

(mi,m2) 

" X! p{mi,m2\a,xi)p{s\a,xi,mi,m2) 

(mi,m2) 

— ^ p{mi,m2\a,xi)p{s\a,mi,m2) 

{nil ,^12) 

= ^ p{mi,m2\a,xi)p{s\a) 

(mi ,7712 ) 

= p{s\a) ^ p{mi,m2\a,xi) = p(s\a), 

(rni,m2) 



where 

(a) follows from the chain rule, 

(6) follows from the fact that Xi = /(A/i), 

(c) follows from the channel model that states that (A/i, A/2) — A — S form a Markov chain. 

Thus, we can say that the following Markov, Xi — {Mi, M2) — A — S, holds. Therefore, we conclude that 

Ps\A,xAs\a,xi) = Ps\A{s\a). 

2) -Py|Xi,A,s,(7,X2(2/|a;i,a,s,u,X2) = PY\XuX2,s{y\xi,X2,s) 

Follows from the fact that the channel output at any time, i, is assumed to depend only on the channel inputs 
and state at time i. 

3) Px2\Xi,s,u,A{^2\xi,s,u,a) ^ Px2\Xi,s,u{x2\xi, s,u) 

This is due to the fact that we take the auxiliary random variable U to be Ui = 

{Xl-\Xl^^„ Ml, A/2, Sf+i, F'-i, A"). 



With this we complete the proof. 
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VI. The Gaussian Action-MAC 



In this section, we examine the Gaussian channel setting of our Action-MAC. We present the channel model and 
the power constraints and obtain a closed formula for the capacity region. Using similar steps to those used in the 
proof of the Gaussian Action-MAC, we also derive the capacity expression for the Gaussian action-dependent point- 
to-point channel. The point-to-point model was introduced in 1111 and was left open. We proceed to discuss the 
results of the Gaussian Action-MAC and observe that the formula for the capacity region found contains previously 
known results, such as the capacity region for the Gaussian Generalized Gel'fand Pinsker (GGP) MAC found in 
||5| as well as the new action-dependent point-to-point result. Furthermore, in Appendix |B] we give an alternative 
proof for the capacity of the action-dependent point-to-point channel, using a one-to-one correspondence to the 
GGP MAC Is) with only a common message. 

Section IVI-AI presents the channel model, the relations between the channel variables, the power constraints and 
the main theorem. In Section IVI-BI we find an upper bound on the capacity region of the Gaussian channel. Section 
IVI-CI shows that the upper bound found can be achieved by taking particular distributions of our random variables, 
thus attaining that our upper bound is, indeed, tight. We conclude with some remarks on different results that can 
be derived from our capacity region. 

A. Capacity and Channel Model 

Consider the following setting. The channel probability is given by the following relation between Xi i, X2,i, Si 
and Yi 



where 

• Z" is an i.i.d Gaussian noise process with zero-mean and E[Zf] ~ N, independent of X and S. 
. The process, 5", is combined out of yl"(Mi, A/a) and i.e. 5" = A"(A/i, A/2) + W". Therefore, we can 
look at the channel output, Yi, as: 



• W" is an i.i.d Gaussian noise process with zero-mean and E[W^] = Q, independent of and X", A". 

• The actions are constrained by 



Yi — Xi^i + X2S + Si + 



(30) 



Y, = Xi,, + X2,^ + A^ + W^ + Z,. 



(31) 




(32) 



• The power constraints on the channel inputs are: 





(33) 



We obtain the capacity region for this channel in the following theorem. 
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Theorem 5: The capacity region of the Gaussian action-dependent MAC, under the power constraints 
is given by the union of rate pairs satisfying 



R2<- log 



N - 



■P2+PA+Q- P2ph - PapIa + 'iVP2PAP2A - ■2y/PlPXpi2PlA + 2^/T^p2w) (a^(P?a " 1) " 

^((pIa - + Q + P2pI„ + 2^rf^p2w) - P2A 



(34) 



1 (N + P1+P2+PA+Q + 2V7V^pi2 + 2v^ftPAPlA + 2V7YPAP2A + 2y^P^p2w) (n{pIa " 1) " ^^a) 
Rl+R2<-log^ -, r ^, (35) 

2 A^((P?A - + + -Papity + 2V7Wp2w) - P2Aj 
for some pi2 € [-1, 1], piA S [-1, 1], P2yi e [-1, 1], p2W e [-1, 1] where 



such that 



A = 1 - p?2 - PlA - pIa - pIw + PiaPw + 2piAP2APl2, 



A > 0. 



(36) 



(37) 



The proof of this theorem is obtained in three stages. First, in the converse subsection, we state two lemmas that 
show that the capacity region dHJ given in Theorem [T] is upper bounded by the region 



i?2 < I{A-Y\Xi) + h{X2\Xi,A,W)-h{X2~ X}r{X^,A,W,X2 + Z)) 



(38) 



R1 + R2 < IiA,Xi;Y) + h{X2\Xi,A,W) ^ hiX2- X^^'\Xi,A,W,X2 + Z)), (39) 

where {Xi, X2, A,W, Z,Y) are jointly Gaussian and Xl''{Xi,A,W,X2 + Z) is the optimal MMSE estimator 
given the measurements (ATi, A, W, X2 + Z). Second, the exact expressions of the upper bound for our region are 
calculated as functions of <j\_^, and a\. Next, we show that replacing and cr\ with Pi, P2 and Pa, 

respectively, further increases the region. 

This solution introduces the constraint (iJTT l which follows from the fact the allowable values of the covariances 
i?[Ari^] — (TiA, E[XiX2] ~ (712, £^[-^^2^] = o'2A and E[X2W] ~ (J2W such that the covariance matrix 

( Pi (J12 ^ 

(T12 P2 O2A CF2W 

A= criA (T2A Pa , (40) 

cr2w go 

iV 
satisfies the nonnegative-definiteness condition, i.e. 

dot (A) - al^al^NPiPA + 2Gi2<JiA<y2ANQ - a^A^PiQ - <j12NPaQ + NP1P2PAQ > 0, (41) 

or, equivalently, as a function of P12, PiA, P2A and p2w 

^ = 1 - P12 - PiA - pIa - pIw + PiapIw + 2piAP2APi2 > 0, (42) 

where 

CTlA 0'12 Cr2A Cr2W 



V 



PlA 



P12 



P2A 



P2W 



(43) 
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Finally, in the direct part, we show that this region is, indeed, achievable. 

The capacity region for the Gaussian Action-MAC setting is plotted in Fig. |4] for various values of Pa- It is 
particularly interesting to notice that by taking P4 = we derive the rate region for the GGP MAC plotted in ||5]- 
We can interpret this as taking ^ = 0, i.e. disregarding the action from our Gaussian channel model reduces our 
model to the Gaussian GGP MAC. In addition, we also see that the rate region increases with Pa- However, we see 
that the region only grows logarithmically fast as a function of Pa, a fact that we can also derive from (|34]|-(|35]|. 




0.2 0.4 0.6 0.8 1 1.2 1.4 



Fig. 4. The capacity region for various values of Pa and Pi = 0.5, P2 = 1, Q = 1-5, = 1. The smallest region con'esponds to Pa = 
and is the exact region of the GGP MAC plotted in (5]. The next regions correspond to Pa = 1, 2, 3, 4, 5, where we can see that the regions 
grow grow as Pa increases, but by smaller increments. 

B. Converse 

In this section, we present a converse to the upper bound for the Gaussian setting of the Action-MAC. We start 
by finding an upper bound to the region found in Theorem [T] and show that it suffices to consider only random 
variables {Xi, X2, A, W, Z, Y) that are jointly Gaussian. Next, we calculate the expressions for the upper bound in 
terms of and ElA^]- We show that replacing the last terms with Pi, P2 and Pa, respectively, further 

increases the upper bound. Furthermore, we add the nonnegative-definite constraint on the covariance matrix of our 
variables such that we obtain an equivalent problem in terms of Pi, P2, Pa, pn, PiA, P2A and p2W- We conclude 
by obtaining a set of equalities that give us a simpler representation for our upper bound. In the direct part, we 
will see that this representation for the upper bound is achievable. 
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We begin by presenting the next lemma, where we show that the following region provides an upper bound to 
the region given in Theorem [T] 

Lemma 6: The closure of the convex hull of the set of rate pairs satisfying 

R2 < I{A:Y\Xi) + I{X2]Y\Xi,A,W) 
Ri + R2 < I{A,Xi;Y) + IiX2;Y\Xi,A,W), (44) 

for some joint probability distribution of the form 

Pa,s,Xi,X2.y = PxiPa\XiPs\aPx2\Xi,s,aPy\Xi,X2,s, 

is an outer bound on the capacity region found in Theorem [T] 

Proof: Recall the equivalent capacity region for Theorem [T] ( fTOt , given in Corollary |3] For the term R1+R2 < 
I{Xi,A,U;Y) + I{Xi,U;S\A) in ^ we have: 

I{Xi,A, U; Y) + I{Xi,U; S\A) = I(Xi,A, U; Y) - I(Xi, U;W + A\A) 

liXi ,A,U;Y)- I{Xi , A, U; W) 
= /(Xi, A; Y) + I{U- Y\X^,A) - /(Xi, A; W) - I{U- W\X^,A) 

I{X^,A- Y) + I{U; Y\X,,A) - I{U; W\X,,A) 
= I{Xi,A; Y) - h(U\Xi,A, Y) + h{U\Xi,A, W) 
= I{Xi,A- Y) + /([/; Y\Xi,A, W) - /([/; W\Xi,A, Y) 
<I{Xi,A-Y) + I{U;Y\X^,A, W) 
= /(Xi, A; Y) + h{Y\X^,A, W) ~ h{Y\X^,A, W, U) 

< I{Xi ,A-Y) + h{Y\Xi ,A,W)- h{Y\Xi , X2, A, W, U) 
I{X^,A-Y) + I{X2;Y\X^,A,W), 

where 

(a) is due to the fact that W and A are independent of each other, 
(6) is due to the fact that W is independent of {Xi, A), 
(c) is due to the fact that conditioning reduces entropy, 
(rf) is due to the properties of the channel. 

The proof for R2 < I{A; Y\Xi) + I{X2; Y\Xi, A, W) is obtained in a similar manner to that above, except that 
for i?2 we have the term I{A:Y\Xi) instead of I{Xi,A;Y). ■ 

Now, that we have an upper bound (l44l i for the capacity region of Theorem[Tl we show in Lemma|7]that choosing 
jointly Gaussian {Xi,X2, A, W, Z, Y) further increases the region. 

Lemma 7: The region given in (l44l i increases by taking (Xi, X2, A, VF, y) to be jointly Gaussian. 
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Proof: Let us take the expression bounding Ri + R2, i.e. I[A, Xi;Y) + /(X2; Y\Xi,A, W): 

I{A,Xi-Y) + I{X2;Y\Xi,A,W) = h{Y) - h{Y\Xi,A) + h{Y\Xi,A, W) - h{Y\Xi,X2,A, W) 

^h{Y)-I{W-Y\Xi,A)-h{Z) 

= h{Y) + h{W\Y, Xi,A)- h{W) - h{Z), (45) 

where (a) follows from the fact that W is independent of {A,Xi). 

We invoke the maximum differential entropy lemma lfT4l . which states that the differential entropy, h(X'"-\Y"), 
for a pair of random vectors, (X", F") ^ /(x", y"), with covariance matrices, Kx and Ky, is maximized for 
jointly Gaussian (X", F"). Hence, the differential entropies of h{Y) and Y, Xi,A) are maximized if we take 
{Xi ,X2,A, W, Z, Y) to be jointly Gaussian. 

In the same manner, for the expression bounding R2, i.e. I{A; Y\Xi) + I{X2\ Y\Xi,A, W), we have 

I{A; Y\Xi) + I{X2;Y\Xi,A, W) = h{Y\Xi) + h{W\Y, X^.A)- h{W) - h{Z). (46) 

Studying the last expression again reveals that it is maximized by taking (Xi, X2, A, W, Z, Y) to be jointly Gaussian. 

■ 

Now we would like to calculate the expressions for fixed second moments 

E[Xl] = < Pi E[Xl] =. < P2 E[A^] ^ < Pa 
E[XiX2] = (712 E[XiA] = aiA E[X2A] = G2A E[X2W] = <J2w (47) 
Furthermore, we have 



= cr^^ + CT^^ + ctI + Q + TV + 2cri2 + Icjxa + 2cr2A + 2(7: 



2W, 



where 



- -2" 

'^W\{Y,Xi,A) = Six - ^12^22 S21, (48) 



511 = E[W^]^Q, (49) 

512 - E[W -{Y^X,, A)] = (Q + a2w, 0,0), (50) 



S21 = E[W-{Y,X,,Af] = (^Q + a2w,0,of, (51) 
E22 = E[(Y,Xi,Af ■iY,Xi,A)], 

2 I I 




Therefore 

i?2 < /ill^l^i) + h{W\Y, Xi,A)- h{W) - h{Z) 
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1 , ,^Y\Xi'^W\Y,Xi,A 

2 1°S( Qf^ ) 



= ^{<^X2^^A^Qi^iPl2,PlA-,P2A,P2w) 



(52) 



and 



Ri+R2< h{Y) + hiW\Y, Xi,A)~ h(W) - h{Z) 

2 2 

= - log ( ■ 1 

2 QN ' 



■ log 



N 



[{pIa ~ + Q + '^x^pIw + 2a/'^X2<3^2w) - <^i, A 



^(C^il : ^X2 ' Pl2, PlA, P2A, P2H--)- 



(53) 



To sum up, our capacity region is upper bounded by the closure of the set that contains all the rates (i?i,i?2) 
that satisfy 

R2 < r((T^^,CT^,(3,iV,pi2,piA,m,P2H') 



for some covariance matrix 



< 




2 

''^X2^ 


a\, Q, N, P12,P1A, P2A, P2w)- 


(54) 
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02A 












(55) 







'^2W 





Q 











[ 











n) 






4i < Pi 


'^i. < P2 


4 < Pa- 







such that 

and the nonnegative-definiteness condition is satisfied, i.e., 

det (A) = alAdly^Naj^yA + 2<t i2(yiA<y 2 aNQ - crl^Naj^^Q - af^Na^Q + Na^^x^^^AQ > 0, 
or, equivalently, as a function of P12, PiA, P2A and p2w 



(56) 



1 - P12 - PlA - pIa - pIw + pIapIw + 2piAP2APl2 > 0. 



(57) 
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The nonnegative-definiteness condition states that all leading principal minors of the covariance matrix need to be 
> 0, but we can easily convince ourselves that the condition ( l56b is sufficient. This constraint is due to the fact that 
any symmetric nonnegative-definite matrix is a covariance matrix. Therefore, by adding this constraint we have a 
one-to-one correspondence between the problem in the form of random variables and the problem in the form of 

^Xi' ^X2' '^A' ^12' PlA, P2A, P2W- 

Next, we need to show that we can replace cr^^, cr^^ and a\ with Pi, P2 and Pa, respectively. Let us assume 
that the region ( l54l i is indeed achievable, a fact that we will prove in the following subsection. Observing the term 
n{aj.^, aj.^, a\, Q, N, pi2, piA, P2A, P2w), which is dependent on <y'x-^ , we can see that replacing (t^^ with Pi 
increases the region. This is due to the fact that il{aj^^, aj^^,<T^,Q, N, pi2, piA, P2A, P2w) increases with cr^^. 
Furthermore, the term r((T^^ , cr^, Q, pi2, piA, P2A, P2w) is unaffected by Pi. 

Now, we would like to show that we can replace crxa with P2. Assume, by contradiction, that the best coding 
scheme is obtained for a < P2 . Given this scheme, we reach a contradiction by showing that the informed 
encoder can utilize it's unused power P2 = P2 — cr^, ■ Consider the following new coding scheme. The informed 
encoder uses the given coding scheme to send its signal with power ctx^, but in addition, it also uses its extra 
unexploited power, P2- to send the signal of the uninformed encoder Thus, this new scheme is equivalent to the 
given coding scheme, but with the addition of sending the uninformed user's signal with power P2 + Pi- However, 
since the region ( l54l l increases with ct^^ , taking cr^^ = P2 + Pi (where the extra power comes from the informed 
encoder, without exceeding the power constraints) increases (|54] |. Therefore taking cr^^ — P2 further increases (l54t . 
Hence, we can replace a^^ with P2. A similar argument applies to taking ct^ = Pa- 

Finally, after establishing the upper bound, we give an equivalent representation for it in the next equaUties, 
(l58Tl-(l59]l. This form of the upper bound is needed for the direct part. There, we will show that we can achieve this 
exact form from the general region of the Action-MAC; hence, we can conclude that the bound is tight. 

I{A;Y\Xi)+I{X2;Y\Xi,A, W) 

= I{A; Y\Xi) + hiX2\Xi,A, W) - h{X2\Xi,A, W, Y) 

= I{A-Y\Xi) + h{X2\Xi,A,W)~h{X2~ Xl''{Xi,A,W,X2 + Z)\Xi,A,W,X2 + Z) 

= I{A, Xx-Y) + h{X2\Xi,A, W) - h{X2 - [PiXi + P2A + 133W + 134X2 + Z))), (58) 

and similarly, 

I{A,Xi;Y) + I{X2;Y\Xi,A,W) 

= I{A,Xi;Y) + h{X2\Xi,A, W) - h{X2 - X'^\Xi,A,W,X2 + Z)) 

= I{A, Xi;Y) + h{X2\Xx,A, W) - h{X2 - (/3iXi + /32A + 133W + ^4X2 + Z))), (59) 

where X2{Xi,A,W,X2 + Z) = E[X2\Xi.,A,W.,X2 + Z] is the optimal MMSE estimator of X2 given 
(Xi, A, X2 + Z), where (Xi, X2, A, Z, Y) are jointly Gaussian, hence it is also the linear optimal estimator, 
i.e. X2 ^ Xf' and 
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X2 — X2"{Xi,A,W,X2+Z) — [ (Jr2 <T2A (y-iw P'. 



( Pi <7lA 0-12 ^ 

(^lA Pa (T2A 

Q (J2W 

\ (712 Cr2A cr2W P2+ N j 



A 

w 
X2 + Z 



(60) 



(/3i, f32, Pa, fSi) are taken to be the coefficients of the optimal linear estimator of X2 given {Xi,A, W, X2 + Z). 

To conclude the converse, in Lemma |6] we gave a general upper bound to the region ( fTOl i. Lemma Q showed that 
it suffices to consider only jointly Gaussian random variables. Next, we calculated the expressions for the capacity 
region and showed that we can replace Uxi' '^Xo '^^'^ '^A ^i'^h Pi, P2 and Pa, respectively. Finally, we derived a 
set of equalities, (|58l)-(l59l), that will be shown to be achievable in the direct part. 



C. Direct Part 

In this section, we will choose specific distributions of our random variables and place them in the capacity region 
( [Tol l, given in Corollary |3] of Theorem [T] We will see that, by this choice, we are able to achieve the upper bound 
(l58Tl-(l59]l found in the converse, concluding that it is, indeed, tight. To do so, let us choose {Xi, X2, A, W,Y) to 
be jointly Gaussian with 



E[X^] = Pi, E[Xi] = P2, E[A^] = Pa. 



(61) 



In addition, we take the auxiliary random variable C/ to be {/ = Xi + X2 + fi^S (the notation will become 
clearer later). Note that ?7 is a function of Xi,X2 and 5* so that the Markov U — {Xi,X2, S, A) - Y holds. Now, 
take the expression for Ri + R2 from i.e. I(Xi,A, U; Y) - I{Xi,U; S\Ay. 

I{Xi,A, U- Yyi{Xi, U- S\A) = I{Xi,A; Y) + I{U; Y\Xi,A) - I{U; W\Xi,A) 
= I{Xi ,A;Y) + h{U\W, Xi,A)- h{U\Y, Xi , A) 

= I{Xi,A; Y) + h{Xi +X2+MA + W)\Xi,A, W) - h{Xi +X2+k{A + W)\Y, Xi,A) 
- I{Xi ,A;Y) + h{X2\Xi,A, W) - h{X2 + ^3W\Y, Xi,A) 



(fc) 



(d) 



IiXi,A; Y) + h{X2\Xi,A, W) - h{X2 + P^W ~ l3^Y\Y, Xi,A) 

I{Xi,A; Y) + hiX2\Xi,A, W) - h{X2 + P3W - /34(X2 + W + Z)\Y, Xi,A) 



IiXi,A; Y) + h{X2\Xi,A,W) - h{X2 - /S^W - MX2 + Z)\Y,Xi,A) 
I{XuA; Y) + h{X2\Xi,A, W) - h{X2 - f^^W - MX2 + Z) - p^X^ - hA\Y, Xi,A) 



I{Xi,A- Y) + h{X2\Xi,A, W) - h{X2 - PiXi - P2A - P^W - Pi{X2 + Z)), 



(62) 



where 

(a) is achieved by submitting U ^ Xi + X2 + P^S, 
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(b) is due to the fact that adding a deterministic term, in this case —I34Y, does not effect the entropy, 

(c) is achieved be replacing Y with Xi + X2 + A + W + Z (according to the channel model) where Xi and A 
are conditioned upon and,therefore, known, 

{d) is due to the fact that we take — /Sa ~ (3^ ~ P4. 

Finally, to show equality (e), let us look at the expression h{X2 - (3iXi - (32A- P^W - (34^{X2 + Z)\Y, Xi, A). 
It is clear that 

hiX2 - PiXi - 132A-I33W - l3i{X2 + Z)\Y,Xi,A) < h{X2 - PiXi - P2A - - MX2 + Z)), 
since conditioning reduces entropy. Now consider 

h{X2 - /3iXi - P2A - PsW ~ !3i{X2 + Z)\Y, Xi,A) 

> h{X2 - - (32A - - !3i{X2 + Z)\Y, X^,A, W) 

= h{X2'[iiXi~(32A~(3:iW~[ii{X2 + Z)\Xi+X2 + A + W+Z,Xi,A,W) 

= h{X2 - l3iXi - P2A - - Pi{X2 + Z)\X2 + Z, Xi,A, W) 

= h{X2 - PiXi - hA - - Pi{X2 + Z)), 

where the last step follows from the fact that we can choose /32, and /34 such that {X2~ PiXi — P2A — fS^W ~ 
P4{X2 + Z)) is the error of the optimal MMSE estimate of X2 given {X2 + Z,Xi,A, W) and thus independent 
of {X2 + Z, Xi,A, W). Therefore, we have the following equality 

hiX2 - PiXi - I32A ~ - (34iX2 + Z)\Y, Xi,A) = h{X2 - PiXi - P2A - p^W ~ MX2 + Z)). 

The direct proof for the bound on R2 is obtained in a similar manner 

Summing up the direct part, we have shown that for this choice of U and by choosing (Xi, X2, A, W^, y) to 
be jointly Gaussian, we achieve the upper bound (l58Tl-(l59]l from the converse section. Hence, we obtain a tight 
expression for the capacity region of this channel. 

D. Action-dependent point-to-point Channel 

The capacity for the action-dependent point-to-point channel was left open in fTTl. An achievable scheme was 
given, but was not shown to be tight. Here, we use similar steps to the proof of the Gaussian Action-MAC to 
find the capacity of the point-to-point channel. Moreover, since the point-to-point channel is a special case of the 
Action-MAC, its capacity expression can be derived directly from the capacity region of the Gaussian Action-MAC 
found in the previous subsection. This is done by considering only the private message, i.e. taking i?i = 0. 

The outline for the direct proof of the capacity expression is given here. An alternative proof is available in 
Appendix |B] In the alternative proof we show a one-to-one correspondence to the previously solved setting of the 
GGP MAC with only a common message IS). After establishing a correspondence, we use the result of the GGP 
MAC to find the capacity of the point-to-point channel. Furthermore, we show that the achievable region found in 
ifTTI is equivalent to the capacity expression and is, indeed, tight. 
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Let us recall the setting of the Gaussian action-dependent point-to-point channel ifTTI . Here, the channel probability 
is given by the equation 

Y, = X, + S, + Z, = X, + A, + W, + Z„ (63) 

where W and Z are defined as before, in addition to the power constraints: 

- y A^{M) <Pa, - y X.,{Mf < Px. (64) 

Now, using similar steps as in the proof of Theorem |5] we obtain a closed capacity expression for the Gaussian 
action-dependent point-to-point channel given in the next corollary. 

Corollary 8: The capacity of the Gaussian action-dependent point-to-point channel is 



^_1^^j {N + Q + Px+Pa + + 2./P^pxw){N + Px[l - pIa - Pxw)) \ 

2°^V N{N + Q + Pxp\w + '^^P^Pxw-Px{1-p\a-p]cw)) ^' 

for some pxA G [0, 1] and pxw G [—1,0], such that 

p\a + Pxw<^^ (66) 

where 

C^XA oxw 
PXA — r, 7 Pxw 



Let us give an outline for the proof, using similar steps as for the proof of the Gaussian Action-MAC. First, 
recall the capacity expression for the general action-dependent point-to-point channel: 

C = I{A,U;Y) - I{U]S\A). (67) 

Taking similar steps to those taken in the converse of the Gaussian Action-MAC setting, we can show that ( |67] | is 
upper bounded by: 

I{A, U; Y) - I{U; S\A) < I{A; Y) + I{X- Y\W, A). (68) 

Second, similarly to Lemma |7] we show that (l68T l is maximized by taking (X, A, W, Z, Y) to be jointly Gaussian. 
The next step is to calculate the expression 

I{A; Y) + I{X; Y\W, A) = h{Y) + h{W\Y, A) ~ h{W) - h{Z) 

= 2 1°S(^^). (69) 
We calculate the expression for fixed second moments 

E[X2] ^a\<Px E[A^] =a\<PA E[XA] = axA E[XW] = gxw (70) 

and show that replacing a\ and a\ with Px and Pa, respectively, further increases the region. Following these 
steps shows that ( |65] | is an upper bound for the capacity of the Gaussian setting. 
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Third, with the next set of equahties we can show that this upper bound is achievable in the same manner as in 
section IVI-CI 

/(A; Y) + I{X; Y\S, A) = I{A; Y) + h{X\A, W) - h{X - X^'"{A, W,X + Z)), (71) 

where W,X + Z) is the optimal linear MMSE estimator of X given {A, W,X + Z). For the direct part, 

we can achieve dTTT i from the general capacity expression by taking U = X + (3S, where S ~ A + W . Therefore, 
we can conclude that the expression ( |65] | is indeed, tight and is the capacity for the Gaussian action-dependent 
point-to-point channel. 

Alternatively, we can also derive (|65t in two additional manners. Firstly, since the point-to-point channel is a 
special case of the MAC, we can derive (|65] | directly from the region ( l35b given in Theorem |5] This is done by 
only considering i?2, i-C- Ri — 0, and taking the following variables to be Pi — > 0, P2 — > Px, Pa — > Pa, 
Q — > Q, N — > N, pi2 — > 0, piA — > 0, P2A — > pxA and p2w — > pxw- Secondly, we show in Appendix|B] 
that we can obtain (l65l l, using the equivalent representation 

r7 = lw (i I Pxi^-PxA-plw) \ I 1, I {VpI + pxaVp^Y 

2 n ^ N n Px{l~pjcA~Pxw) + i'^w+PxwVP^y+N^ 

The representation (l72t is simply obtained by reorganizing ( |65] ). We prove the capacity of the action-dependent 
point-to-point channel is equal to the equivalent representation (|72] | by using a one-to-one correspondence to the 
GGP MAC Is) with only a common message. 

VII. Duality Channel-Source Coding with Action 

In this section we recognize and discuss the duality between the MAC with action-dependent state information at 
one encoder and the "Successive Refinement with Actions" rate distortion setting. The information-theoretic duality 
between channel coding and source coding was first recognized by Shannon in his milestone paper ifTTl . Chiang 
and Cover ifTsl . as well as Pradhan, Chou and Ramchandran lfT9l . further expanded the discussion of duality to the 
basic settings with side information. In addition, duality for source and channel coding with action-dependent side 
information was noted by Kittichokechai, Tobias, Oechterin and Skoglund ll20l . 

Source coding with side information was presented by Wyner and Ziv in their landmark paper 11211 . They discussed 
the case where side information is available noncausally at the decoder and not available at the encoder This case 
is dual to the Gel'fand-Pinsker ||2| channel coding problem, as shown in lITSl and |fT9ll . Operational duaUty between 
these two setting was shown by Gupta and Verdu 1221 . Rate distortion problems for two decoders, where only one 
of them has access to correlated side information were considered by Kaspi 1231 . as well as by Heegrad and Berger 
II24I . This model was extended by Steinberg and Merhav in ll25l to successive refinement with side information. A 
successive refinement model with conditionally less noisy side information was studied by Timo, Oechtering and 
Wigger in ll26l . 

The novel idea of source coding where the system can take actions that affect the availability, quality or nature 
of the side information was introduced in Weissman et al. in ll27l . They presented the rate distortion setting with a 
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side information vender at the decoder This setting was shown to be dual to Weisman's action-dependent point-to- 
point channel in ll20l . The "Successive Refinement with Actions" rate distortion setting, introduced in lfT3l . is an 
extension of these previously discussed rate distortion settings to the case where we have two decoders, where only 
one has access to vender side information. Additionally, the encoder transmits the source sequence to the decoders 
with a common rate and a private rate. This setting is illustrated in Fig. |5] 

Here, we characterize the dual relationship between the "Successive Refinement with Actions" rate distortion 
setting and the Action-MAC channel coding setting. We start by revising the "Successive Refinement with Actions" 
rate distortion model as well as the theorem for the rate region Tl{Di, D-z). Next, we state the equivalence 
relationships of these channel coding and rate distortion problems and give a set of duality principles that show the 
connections between them. We end this section with a detailed discussion on the duality between the two settings; 
this is followed by additional duality relationships that are deduced as special cases. For instance, we recognize the 
duality between MAC with state information at one encoder Q and a special case of the "Successive Refinement 
for the Wyner-Ziv problem" ||251 . 



Ti(X") e {1,2,..., 2"-"!} 



Encoder 



T2{X") e {1,2,. ..,2"-"=} 



Uninformed 
Decoder 



Informed 
Decoder 



Vender 

p{s\a, x) 



X" 



X^ 
— ► 



Fig. 5. The "Successive Refinement with Actions" rate distortion setting, presented in 1131 



A. The "Successive Refinement with Actions" rate distortion setting 

Let us revise the formal definition of the dual rate distortion problem. We consider a source sequence 
Xi,X2,...,Xn of i.i.d. random variables where each variable is drawn ~ p{x) from a source alphabet X. Let 
Xi and X2 denote the reconstruction alphabets and consider two, single letter distortion measures 



dj : X yc Xj M., j = l,2. 



(73) 
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The distortion between n-sequences in A" x Xj is defined in the usual way: 

1 " 

x"]) = -Y. for J = 1, 2. (74) 

1=1 

Definition 2: A ((2"^^, 2"^^), n) rate distortion code consists of an encoder and two sets of decoders, one which 
receives the message at rate i?i and one which receives the message at rates i?i,i?2- The code is defined by encoding 
functions 

Ti : A-" ^ {l,2,...,2"^i} 

T2 : A-" ^ {1,2, ...,2"-^^^}, (75) 

an action strategy 



and decoding functions 



/ : {l,2,...,2"^^nx{l,2,...,2"^^n^-4", (76) 
5i.„ : {l,2,...,2"«i}^<Yf 

g2,n ■■ 5"x{l,2,...,2"«i}x{l,2,...,2"^^}^i'2" (77) 

Definition 3: A quintuple (i?i , R2, Di, D2,C) is said to be achievable if, for all e > and for a sufficiently 
large n, there exists a ((2"^i , 2"^^), ri) code satisfying 

n 

E[J2d{X^,XJ,^)]<n{D + e) for j = l,2 (78) 



4=1 

and 

n 

E[Y^A{A,)\<n{C + e), (79) 
j=i 

where A is the cost function. 

Definition 4: The rate distortion region TZ{DiD2) is defined as the closure of the set for all achievable rate 
distortion quintuples {Ri, R2, Di, D2,C)- 

The rate region TZ{DiD2) for the "Successive Refinement with Actions" setting, illustrated in Fig.|5] was proven 
in ifTSl and is the closure of the set of all the rate tuples R2) such that, 

R1+R2 > I{X;Xi) + I{A;X\Xi)+I{X;U\S,A,Xi), (80) 

for some joint distribution 

P{x, a,u, s,xi) — P{x)P{a, u, xi \x)P{s\x, a), (81) 

such that 

E\di{X,Xi)] < Di, (82) 
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E\d2{X,X2{S,U))\ < D2, 

e\a{A)\ < c. 



(83) 
(84) 



B. Duality results between the Action-MAC and the "Successive Refinement with Actions" 

Using a set of simple duality transformation principles we obtain a precise characterization of the functional 
duahty between the Action-MAC and the "Successive Refinement with Actions" rate distortion setting. The duahty 
transformation principles between the two settings are given in Table HI] In other words, for a given channel coding 
problem we obtain a rate distortion problem and vice versa. Here, the roles of encoder and decoder are functionally 
interchangeable and the input-output joint distribution is equivalent with some renaming of variables. 



TABLE II 

Duality transformation principles between the "Successive Refinement with Actions" rate distortion setting and 

the action-mac 



"Successive Refinement with Actions" 


Action-MAC 


R>R{Di,D2) 


B<C 


Decoder inputs / Encoder outputs: 
Ti e {1,2, ...,2"-f*i} 
T2 e {1,2,...,2"«2} 


Encoder inputs / Decoder outputs: 
Ml 6 {1,2, ...,2"-^i} 
M2 e {1,2, ...,2"«2} 


Decoder outputs / Source reconstruction: 


Encoder outputs / Channel input: 


Encoder input / Source: 
X" 


Decoder input / Channel output: 


Decoder functions: 
91 :{l,2,...,2"«i}^;fi" 
g2 ■■ {l,2,...,2"-f'i} X {1,2, ...,2"«2} x 5" X^^ 


Encoder functions: 
/i : {l,2,...,2"^i} -5. A'l" 
/2 : {1, 2, 2"^i } X {1, 2, 2"^2 }x S" ^ Xl^ 


Action strategy: 
/a : Ti X r2 ^" 


Action encoder: 
/a ; >!i X A^2 .4" 


U auxiliary random vaiiable 


IJ auxiliary random variable 


S side information 


S state information 


Joint distribution P{x, a, s, xi , u, £2) 


Joint distribution: P(a, s, x\, u, X2, y) 


Markov S - {A, X) - U, Xi 


Markov S ~ A - Xi 



TABLE III 

Corner points (_Ri , _R2 ) of the Action-M AC setting and its dual rate distortion setting 









Ri 






R2 




Rate 


/(X;Xi)H 


h/(A;X|Xi) 




- I(S;U\A,Xi) 









Distortion 






I(Xr,X) 




I{A;X\Xi)- 


\-I{U;X\XuA) 


- I{U;S\Xi,A) 


Action 


l{Xi;Y)- 


|-/(A;y|Xi) 


+ I{Y;U\A,Xx) 


-I{S;U\A,Xi) 









MAC 






I(Xi;Y) 




IiA;Y\Xi)- 


\-I{U;Y\Xi,A) 


- I{U;S\Xi,A) 



"Successive Refinement with Actions" 
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MAC with Action-Dependent State Information 
at One Encoder 



Uninfonned 
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Infonned 
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S" 



A"{Mi,M2) 



Vender 

p(s\a. x) 



► 



MAC 

y" I iXh.n'h) 

p(y|:ri,T2,s) ► Decoder — ► 



S") 



p(s\a) 




HX:X,)+HA:X\X,)+nX:U\A.X,)-HS;U\A,X,) HXr,Y) + I{A:Y\X,) + I(Y;U\A.X,) - I(S:U\A.X,) 



Fig. 6. A summary of the duality results. In the figure we see that if we take out the channel block from the Action-MAC model, we have 
an exact mirror reflection of the two settings. In this mirror reflection the roles of encoder and decoder are functionally interchangeable. The 
comer points are of the same form, with an exchange of variables X <r^ Y and Xi ^ Xi. The bound on Ri + R2 is also of the same form, 
with the same exchange of variables and with the additional exchange of >-f><. 



To further investigate the duality between the settings, we examine their regions' corner points. Observing the 
resemblance between the regions' corner points helps us gain better intuition regarding their correspondence. The 
resemblance is not clear from simply looking at the expressions of both regions. In order to see the similarities, 
we need the following lemma. 
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Lemma 9: Let S — {A,X) — {U,Xi) be a Markov chain. Then the following equality holds: 

I{X; Xi) + I{A; X\Xi) + I{X; U\S, A,Xi)= I{X; X^) + I{A; X\Xi) + I{X; U\A, X^) - I{S; U\A, X^) (85) 

Proof: Consider the term 

I{X;U\S,A,Xi) ^ I{S,X;U\S,A,Xi) - I{S;U\A,Xi) 

= I(X; U\S, A, Xi) + I{S; U\X, A, Xi) - I{S; U\A, X^) 
=I{X- U\A, Xi) - I{S] U\A, Xi) 

where the last equality is due to the Markov property. ■ 
Using the last lemma, we can write the rate region for our rate distortion setting as 

i?i > I{X-Xi) 

R1+R2 > I{X;Xi) + I{A;X\Xi)+I{X;U\A,Xi)- I{S;U\A,Xi). (86) 

Let us recall the capacity region for our Action-MAC setting ( fT3] l given in Corollary |3] 

R2 < I{A:Y\Xi)+I{Y;U\A,Xi)-I{S;U\A,Xi) 
R1+R2 < I{Xi;Y)+I{A;Y\Xi)+I{Y;U\A,Xi)-I{S-U\A,Xi). 

Looking at these two regions we notice equivalent relationships between the Action-MAC capacity region and 
the rate distortion rate region. Using the duality transformation principles presented in Table |II] we see an evident 
connection between the formulas bounding the rate sum i?i +i?2- This is done by consistently renaming the variables 
in ( |86] |. X ^ Y and Xi <-> Xi. Furthermore, by looking at these expressions for the regions of the two settings, 
we observe the duality of the regions' comer points. The corner points are displayed in Table |lll] 

A graphical summary of the duality results we have obtained is displayed in Fig. |6] 

C. Remarks and special cases 

Throughout the paper we have seen that our Action-MAC setting generalizes two settings previously solved: the 
action-dependent point-to-point channel ifTTI and the MAC with state information available at one encoder setting 
(known as the GGP MAC) 13]. Similarly to the Action-MAC, its dual setting, the "Successive Refinement with 
Actions", also generalizes previously solved rate distortion settings. These setting are the "rate distortion when side 
information my be absent" model, considered by Heegard and Berger ll24l as well as by Kaspi 1231 , a special 
case of the "Successive Refinement Wyner-Ziv" problem ||251 and source coding with side information "vending 
machine" ll27ll . In this subsection, we recognize the duality between the special cases. 

1 ) Duality between the Heegard-Berger/Kaspi rate distortion setting to a MAC with common message and state 
information: 
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Fig. 7. The Heegard-Berger/Kaspi rate distortion setting (on the left) compared to the common message MAC with state information(on the 
right). If we ignore the channel block in the MAC, we have an exact mirror reflection between the settings. In this mirror reflection, the roles 
of the encoders and decoders are substitutable. 



TABLE IV 

Duality results of the Heegard-Berger/Kaspi rate distortion setting vs. MAC with common message and state 

INFORMATION AT ONE ENCODER 



Heegard-Berger/Kaspi 
rate distortion setting 


MAC with common message 
and state information at one encoder 


R > R{Dx,D2) 


R<C 


Rate distortion function 
R{D) = I{Xi;X) + I{U;X\Xi) - I{U;S\Xi) 


Capacity expression 
C = I{Xi;Y) + I{U; Y\Xi) - I(U; S\Xi) 


Decoder inputs / Encoder outputs: 

Te {1,2, ...,2"-f'i} 


Encoder inputs / Decoder outputs: 

Ai e {1,2, ...,2"-f'i} 


Decoder outputs / Source reconstruction: 

X^, 


Encoder outputs / Channel input: 

X^, X^ 


Encoder input / Source: 

X" 


Decoder input / Channel output: 


Decoder functions: 
gi : {1,2, ...,2"^i} 
92 : {1, 2, 2"«i } X 5" X'^ 


Encoder functions: 
/i : {l,2,...,2"-f'i} -> A'f 
/2 : {1,2, ...,2"-f*i} xS" 


U auxiliary random variable 


U auxiliary random variable 


S side information 


S state information 



The first special case duality we recognize is between the well known Heegard-Berger/Kaspi rate distortion 
setting and the MAC with common message and state information at one encoder, solved in 0. We establish this 
duality by using similar duality principles as those given in Table Ull i.e. we see an interchangeable relationship 
between the encoders and decoders in both settings, as well as the variable transformation X ^ Y and Xi <-> Xi. 
Given these transformations, we look at the rate distortion function for the Heegard-Berger/Kaspi model and the 
capacity expression for the MAC. For the rate distortion function, we use the representation given in ||23l . and for 
the capacity expression we us the common message capacity for the GGP-MAC given in fS]- 
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(Ml, Mi) 




Fig. 8. The special case of the Stienberg Merhav rate distortion setting (on the left) compared to the GGP MAC (on the right). Once again, 
taking out the channel block in the MAC, we have an exact mirror reflection between the settings. 



2} Duality between the GGP MAC and the Stienberg-Merhav rate distortion setting: Using the same set of rules 
for the duality transformation principles obtained in Table we can establish a duality between the GGP MAC 
and the Stienberg-Merhav rate distortion setting. These results are a straightforward extension of the duality results 
of the Action-MAC and its dual rate distortion model, obtained by dismissing the action A. 



TABLE V 

Corner points , i?2) of the GGP MAC setting and the Stienberg-Merhav rate distortion setting 
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I{Xv,Y) + I{Y;U\Xi)- 


-I{S;U\Xi) 





MAC 


I{XrX) 




I{U;Y\Xx)-I{U;S\Xx) 


Stienberg- 


I{X;X-i_) + I{X;U\A,Xi) 


-I(S;U\Xi) 
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7(C/;X|Xi)-/(!7;S|Xi) 
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r < — > 
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p(y\x.s) 



Decoder 
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Fig. 9. The action-dependent point-to-point channel (on the right) compared to the source coding with side infomiation "vending machine" at 
the decoder (on the left). Again, we see that if we take out the channel block from the point-to-point model, we have an exact miiTor reflection 
between the two settings. 
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3 ) Duality between the point-to-point action-dependent channel and source coding with side information "Vending 
Machine ": 

TABLE VI 

Duality results of the action-dependent point-to-point channel vs. source coding with side information "vending 

MACHINE" 



Action-dependent 
point-to-point channel 


Source coding with side information 
"vending machine" 


Capacity expression 
C = I{A; Y) + I(U; Y\A) - I(U; S\A) 


Rate distortion function 

R{D) = I{A; X) + I{U; X\A) - I{U; S\A) 



The duality between these two settings was noted first in ll20l . Similarly to the previous compression, the duality 
transformation principles presented in Table |II] are used to determine the duality between the action-dependent 
point-to-point channel and the source coding with side information "vending machine" model. It is particularly 
interesting to observe that the capacity expression and the rate distortion function are equivalent with some renaming 
of variables, as presented in Table |VT] 

VIII. Conclusions 

In multi-user communication systems today, there is an increasing demand for high data rates. Therefore, it is 
essential to find the fundamental limits of channel models in order to benefit from the channel structure. Motivated 
by this growing need, we extended the study of the MAC. We consider a MAC with two encoders: an encoder 
which has access to a common message and an encoder which has access to a common message as well as a private 
message. This second encoder can take an action, dependent on both of the messages, that affects the formation of 
the states. We find a single letter characterization of the capacity region for this channel. This is done by using a 
three-stage binning coding scheme. We focus specifically on the Gaussian case, find an characterize the capacity 
region and plot some computed results. In the process of this analysis, we find the capacity expression for the 
Gaussian point-to-point action-dependent channel, which was left open in l llj and is independently reported in 
lfT2l . In addition, we explore the dual rate distortion for our Action-MAC model. We give duality principles that 
enable us to effectively estimate the result of one problem, given the result of the other Using the insight gained 
by this duality, we discover further duality comparisons between previously solved settings in source coding and 
channel coding. 

On a final note, we remark that finding a closed, single letter expression of the capacity region leads us to believe 
that such solutions exist for various extensions of this model. One practical extension is a MAC where the informed 
encoder has access to a noisy observation of the channel state, S"", instead of the noiseless state sequence S*". Such 
a setting can model a pubic relay, where the informed encoder observes a noisy version of the pubic relay's output. 
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Another possible extension is where the non informed encoder takes the action rather than the informed encoder 
This setting corresponds to the case that the non informed encoder influences the pubUc relay, while the pubUc 
relay is observed (possibly with noise) by the informed encoder. 

Appendix A 
Proof of Lemma|2] 

Let us prove that the capacity region described in Theorem 1, given in (O, is convex. 
Proof: Let P^, i = 1, 2, 3 be three distributions of the form (|9), i.e.: 

Pi{a,s,u,Xi,X2,y) = P^{xi)Pi{a\xi)Pi{s\a)Pi{u\s,a,Xi)P,{x2\xi,s,u)Pi{y\xi,X2,s). 

The given distributions induce the quantities: 

(/,(;7;r|Xi),/,(t/;5|A,Xi),/,(Xi,[/;y),/,(Xi,[/;5|A)), (87) 

for i = 1,2,3, respectively. Let 7 be a number characterized by < 7 < 1 and 7 = 1 — 7, 
where P3 = 7P1 + 7P2. In addition, let Q be a binary random variable, where P{q = 1) = 7 and 
P{q = 2) = 7, independent of A, S,U, Xi, X2- The distribution of A, S,U, Xi, X2 is according to Pi when 
q ~ 1 and according to P2 when q = 2. Furthermore, we denote U = {U,Q). Note that by marginahzing 
P{xi)P{a\xi)P{s\a)P{u\s, a, xi)P{x2\xi, s, u)P{y\xi, X2, s) over Q, we obtain P3 in the form of (|9]l. This is due 
to the fact that we can write P{u\s,a,xi) equivalently as P{q)P{u\q, s, a,xi). Finally, from the set of equalities: 

jhiU;Y\Xi)+^l2{U;Y\Xi)^h{U;Y\Xi,Q) = h{U;Y\Xi), 

-,hiU;S\Xi,A)+^l2iU;S\Xi,A)=hiU;S\Xi,A,Q) = h{U; S\Xi, A), 

jhiXi,U;Y)+Jl2iXi,U;Y)=hiX,,U;Y,Q) = hiX,M;Y\A), 

^Ii{Xi,U;S\A)+^l2iXi,U;S\A)^hiXi,U;S\A,Q) = h{Xi,U-S\A), 

we derive that region is, indeed, convex. 

■ 

Appendix B 

Alternative Proof to the Gaussian Action-Dependent Point-to-Point Channel 

Weissman lITTI first introduced the Gaussian channel model for the action-dependent point-to-point channel 
illustrated in Fig [TO] He derived an achievable scheme for a lower bound on the capacity of the channel. We would 
like to show that the expression (|72] | is the capacity of this setting and that the lower bound presented in ifTTI is 
an equivalent expression. In order to do so, we look at the Gaussian channel model presented in ||5], namely the 
Gaussian Generalized Gel'fand-Pinsker (GGP) MAC. We examine the case where only a common message is sent 
over the channel. In this setting, we have an informed encoder which has access to the common message in addition 
to knowledge of the channel states non-causally. We also have an uninformed encoder in the sense that it does not 
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know the channel states. This setting is illustrated in Fig[TT| The capacity region for this model was solved in [jS). 
We use the result obtained in 15] to prove the capacity of the action-dependent point-to-point channel. We note that 
similar results to this alternative proof were obtained simultaneously and independently by Choudhuri and Mitra in 
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Fig. 10. Point-to-point channel with action-dependent states. 
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Fig. 1 1 . MAC with state infomiation at one encoder and a common message 



A. Alternative Proof of the Capacity of the Action-Dependent Point-to-Point Channel 

The main idea of the proof is to show that we can obtain a one-to-one correspondence between the action- 
dependent point-to-point channel introduced in ifTTl and the MAC with state information known to one encoder 
channel, presented in ijS). 
Proof: 

We briefly recall the channel model presented in ||5]- The channel is given by: 

Y, = Xl^, + X2^^ + W, + N,. (88) 
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Here, the states are denoted as where the noise processes Wi and Ni are zero-mean Gaussian i.i.d. with 
E[Nf] = N and E[W^] = Q. Individual power constraints are considered: 

-E^M^^i' -E^2.<^2. (89) 

i—1 i—1 

Now, we would like to show that the setting of the Gaussian action-dependent point-to-point channel, illustrated 
in Fig [To] , is a special case of the Gaussian GGP channel with only a common message, illustrated in Fig. [TT] 
Let us take X"(A/) as yl"(A/), and X2{M) as X{M). We can look at the block of the "Action Encoder" as the 
"Uninformed Encoder" and the "Channel Encoder" as "Informed Encoder". 



Action-dependent p-t-p channel 


GGP channel with common message 


A" 

fx ■■MxS'' ^ A"" 





fA-.M^ A'' fx,:M^ X^ 

fx-MxS'' ^ -Y" ^ fx,:Mx 5" ^ X^ 

Notice that we do not lose any of the properties of the setting by looking at the action-dependent point-to-point 
channel as a Gaussian GGP MAC. Both the "Channel Encoder" and "Informed Encoder" have knowledge of S*" 
and the channel inputs and the channel outputs are the same. The covariance matrix I^Xi.X2, s.n becomes: 
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(90) 



Conversely, we show that the Gaussian GGP channel, with only a common message, is a special case of the 
Gaussian action-dependent point-to-point channel. Let us take A"{M) as and X{M) as X2{M). So, we 

can look at the "Uninformed Encoder" block as the "Action Encoder" and the "Informed Encoder" block as the 
"Channel Encoder". 

fx,--M^ XI' ^ fA-.M^A" 
fx,:MxS"^ X^' ^ fx -Mx S" A"" 

In conclusion, we obtained a one-to-one correspondence between the Gaussian action-dependent point-to-point 
channel and the Gaussian GGP channel with only a common message. The capacity region for the Gaussian GGP 
channel was found in IS) and is equal to: 
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(91) 



where 



Pl2 = 



_0\2_ 



P2W — 



(92) 



P12 + P2W < 1- 



(93) 



Hence, the capacity for the Gaussian action-dependent point-to-point channel can be achieved by substituting the 
following transformations in the Gaussian GGP capacity expression. Here, we have M2 = 0, thus R2 = Q, Pi ^ Pa, 
P2 — > Px, P12 — > PxA and p2w Pxw- Substituting these expressions into (|9T] | we get 



^1 < ^log(l 



where 



Px{l- PxA- Pxw)\ , 1 



N 



PXA 



crxA 



iVPX + PriVP^y 



Px{l - p\a - P\w) + i'^w + PxwVPx? + N 



Pxw 



<^xw 



PXA + PXW < 1- 



). (94) 

(95) 
(96) 



B. Remarks 

Recall the result for the achievable rate introduced in Weissman's paper ifTTl for the Gaussian point-to-point 
channel with action-dependent states 

1^ [(1 + 2a)PA + Fx + 1 + (1 + 2,)Q][1 + (P. ja^PA + ,^Q))] X 

2 ^ V(a,7):a=PA+7^Q<Px Px - Pa + I + Q {1 + J 

We would like to show that our result, (|72] |. is consistent with the result obtained in ifTTl . Here, X is defined as: 

X ^aA + "fW + G (98) 
where a^p^ + j^Q < and G - iV(0, Px - (a^P^ + 7^Q)). Therefore we have: 

crxA = aPA 

<yxw = iQ 

OXA VPa 
PXA = „ = a- 



VPlP^ Vp^ 
axw ow 



Notice that now the constraint 



PxA + Pxw < 1 
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becomes 

a^PA + l^Q < Px- 

Now, substituting the expressions in ( |99] l into the capacity region ( |72] |. we achieve the lower bound ( |97| l found 
in ifm and have shown that it is, indeed, tight. Therefore, the capacity expression in ( |97] l is equivalent to the 
expression in ( |72] l. 
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